Multiframe Motion Coupling via Infimal Convolution Regularization for Video Super Resolution

The idea of video super resolution is to use the temporal information of seeing a scene from many slightly different viewpoints in the successive frames of a video to enhance the overall resolution and quality of each frame. Classical energy minimization approaches first establish a correspondence of the current video frame to several of its neighbors and then use this temporal information to enhance it. In this paper we propose the first variational super resolution approach that computes several super resolved frames in one joint optimization procedure by incorporating motion information between the high resolution image frames themselves. As a consequence, the number of motion estimation problems grows linearly in the number of frames, opposed to a quadratic growth of classical methods. In addition, we use infimal convolution regularization to automatically determine the reliability of the motion information and reweight the regularization locally. We demonstrate that our approach yields state-of-the-art results and even is competitive with learning based approaches that require a significant amount of training data.